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Summary 


Recent changes in the policy environment have led states and districts in the Regional 
Educational Laboratory (REL) Northeast & Islands Region to increase the rigor of their 
teacher evaluation systems by including more frequent observations or student test score 
data. States and districts nationwide began reforming their evaluation systems as early as 
2006 and were further spurred to action by federal programs such as the Teacher Incen- 
tive Fund and the 2009 Race to the Top grant program. As of 2012, 30 states required 
that evaluations include evidence of student learning, 25 states required differentiation 
of teacher ratings into more than two categories, and 39 states required annual classroom 
observations (National Council on Teacher Quality, 2012). 

A previous REL Northeast & Islands study of districts in the early stages of implementing 
new teacher evaluation systems found a relationship between school professional climate 
— specifically principal leadership, teacher influence on school policy, and trust — and 
teachers’ support for and fidelity of implementation of the new systems (Riordan, LacirenO' 
Paquet, Shakman, Bocala, & Chang, 2015). Given the many challenges in implementing 
a new evaluation system, members of the Northeast Educator Effectiveness Research Alli- 
ance wanted to explore what factors contribute to teachers’ satisfaction with new evalua- 
tion processes. 

This study examined associations between teachers’ perceptions of school profession- 
al climate and their satisfaction with the evaluation process. It used the responses of a 
nationally representative sample of teachers from four linked questionnaires from the 
National Center for Education Statistics 2011/12 Schools and Staffing Survey and 2012/13 
Teacher Follow-up Survey. 

Key findings are: 

• Most teachers reported being satisfied with the evaluation process: 79 percent 
reported strongly agreeing or somewhat agreeing that they were satisfied with the 
teacher evaluation process in the previous year. 

• Teachers reported positive perceptions of their principal’s leadership but perceived 
themselves to have little influence over school policies. On average, teachers 
somewhat agreed that their principal provides positive leadership (mean of 3.1 on 
a scale of 1, strongly disagree, to 4, strongly agree) but that they had minor influ- 
ence over school policies (mean of 2.2 on a scale of 1, no influence, to 4, great deal 
of influence). 

• Of the two concepts used to measure school professional climate — principal 
leadership and teacher influence — only principal leadership was associated with 
teachers’ satisfaction with the evaluation process. Teachers with more positive per- 
ceptions of their principal’s leadership were more likely to be satisfied with the 
evaluation process. 

• Teachers who were rated at the highest level on their evaluation in 2011/12 
expressed more satisfaction with the evaluation process that year than teachers 
who were rated at lower levels. 

• Teachers whose evaluation process included student test scores were 2.5 times less 
likely to be satisfied with the evaluation process than teachers whose evaluation 
process did not include student test scores. 
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Why this study? 


Recent changes in the policy environment have led states and districts in the Region- 
al Educational Laboratory (REL) Northeast & Islands Region to enhance their teacher 
evaluation systems with more frequent observations or the inclusion of student test score 
data. Federal grant programs, such as the Teacher Incentive Fund and Race to the Top, 
and policies, such as federal Elementary and Secondary Education Act flexibility waivers 
(U.S. Department of Education, 2013), offer incentives for states to change their evalua- 
tion systems. As of 2012, 30 states nationwide required that evaluations include evidence 
of student learning, 25 states required differentiation of teacher ratings into more than two 
categories, and 39 states required annual classroom observations (National Council on 
Teacher Quality, 2012). As of August 2014, 43 states, the District of Columbia, and Puerto 
Rico received federal Elementary and Secondary Education Act flexibility waivers, which, 
among other things, required enhancing teacher evaluations if the state had not already 
done so. 

Given the challenges of implementing new teacher evaluation systems, education leaders 
want to understand teachers’ perceptions of the new systems, as well as the factors related 
to successful implementation of evaluation systems. Research suggests that the reactions of 
those affected by a new policy influence its successful implementation (Datnow & Castel- 
lano, 2000; McLaughlin, 1987). One area of interest is how new teacher evaluation systems 
are related to school professional climate (see box 1 for definitions of key terms in the 
report). Since many new evaluation systems include multiple measures of teacher perfor- 
mance, successful implementation might be related to teachers’ perceptions of their princi- 
pal’s leadership and their own influence over their school’s policies and practice. 


Given the 
challenges of 
implementing new 
teacher evaluation 
systems, education 
leaders want 
to understand 
teachers’ 
perceptions of 
the new systems, 
as well as the 
factors related 
to successful 
implementation of 
evaluation systems 


This study builds on a previous REL Northeast & Islands study in New Hampshire that 
identified elements of a strong school professional climate — principal leadership, teacher 
influence, and trust — as positively related to teacher support for a new evaluation system 
(Riordan et al, 2015). That study also found that teacher influence was related to the 
fidelity of implementation of the new system. Northeast Educator Effectiveness Research 
Alliance members wanted to further explore the relationship between school profession- 
al climate and teachers’ satisfaction with the evaluation process. (For the literature base 
that informs this study, see appendix A.) In addition to addressing the alliance’s research 
agenda, this study is intended to inform policymakers and education leaders in the REL 
Northeast & Islands Region and nationally about the relationships between school profes- 
sional climate and teacher evaluation systems. 


What the study examined 


The study examined teachers’ perceptions of their school’s professional climate and their 
satisfaction with the process by which they were evaluated. The data were from a nation- 
ally representative sample of teachers who responded to questionnaires that are part of the 
National Center for Education Statistics Schools and Staffing Survey and Teacher Fol- 
low-up Survey. 

Five research questions were addressed: 

• To what extent do teachers’ responses on the Teacher Follow-up Survey indicate 
that they were satisfied with the evaluation process? 


1 


Box 1. Key terms 


Current teacher. A respondent who was employed as a teacher during the 2011/12 and 2012/13 school years, 
although not necessarily at the same school. 

Former teacher. A respondent who was employed as a teacher during the 2011/12 school year but left the teaching 
profession, as reported on the 2012/13 Teacher Follow-up Survey Questionnaire for Former Teachers. 

Principal leadership. A composite variable from teachers’ responses on a four-point scale indicating level of agree- 
ment (strongly disagree, somewhat disagree, somewhat agree, and strongly agree) on six items related to their 
principal’s leadership on the National Center for Education Statistics 2011/12 Schools and Staffing Survey Teacher 
Questionnaire: 

• The school administration’s behavior toward the staff is supportive and encouraging. 

• The principal enforces rules for student conduct and backs me up when I need it. 

• The principal knows what kind of school he or she wants and has communicated it to the staff. 

• In this school, staff members are recognized for a job well done. 

• I am generally satisfied with being a teacher at this school. 

• I like the way things are run at this school. 

Satisfaction with evaluation process. Teachers’ level of agreement (strongly disagree, somewhat disagree, some- 
what agree, and strongly agree) with the following statement on the National Center for Education Statistics 
2012/13 Teacher Follow-up Survey (Questionnaire for Former Teachers or Questionnaire for Current Teachers) about 
the process for evaluating their work as teachers during the previous school year (2011/12): I was satisfied with the 
formal evaluation process. 

School characteristics. Enrollment, urbanicity, percentage of students who are English learner students, percent- 
age of students approved for free or reduced-price lunch, percentage of students with an individualized education 
program, and percentage of students who are a racial/ethnic minority. 

School professional climate. A rating generated by the study team that is based on teachers’ survey responses to 
items on the National Center for Education Statistics 2011/12 Schools and Staffing Survey Teacher Questionnaire 
asking about the working environment for school professionals and that specifically focuses on principal leadership 
and teacher influence. See definitions for principal leadership and teacher influence and appendix B for more on the 
survey items for these scales. 

Teacher influence. A composite variable from teachers’ responses on a four-point scale (no influence, minor influ- 
ence, moderate influence, and a great deal of influence) to seven items on the National Center for Education Statis- 
tics 2011/12 Schools and Staffing Survey Teacher Questionnaire: 

How much actual influence do you think teachers have over school policy AT THIS SCHOOL in each of the follow- 
ing areas? 

• Setting performance standards for students at this school. 

• Establishing curriculum. 

• Determining the content of in-service professional development programs. 

• Evaluating teachers. 

• Hiring new full-time teachers. 

• Setting discipline policy. 

• Deciding how the school budget will be spent. 

Teacher demographic characteristics. Gender, Hispanic ethnicity, and a three-category race variable (Black, White, 
or other). 

Teacher professional characteristics. School level taught (elementary or secondary), years of teaching experience, 
self-reported previous-year evaluation rating, and teaching status (current or former teacher). 
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• What are teachers’ perceptions of school professional climate (that is, principal 
leadership and teacher influence), as measured by the Schools and Staffing Survey? 

• Is teachers’ satisfaction with the evaluation process related to how they perceive 
the school professional climate? 

• Is teachers’ satisfaction with the evaluation process related to teacher demographic 
characteristics, teacher professional characteristics, or school characteristics? 

• Is teachers’ satisfaction with the evaluation process related to whether the evalua- 
tion process included student test scores? 

Data sources and methods are described in box 2, and more detail is provided in appendix B. 


Box 2. Data sources and methods 

Data sources 

The data for this study are from four linked survey questionnaires of public schools and teach- 
ers that are part of the National Center for Education Statistics 2011/12 Schools and Staff- 
ing Survey (School Questionnaire and Teacher Questionnaire) and 2013/14 Teacher Follow-up 
Survey (Questionnaire for Former Teachers and Questionnaire for Current Teachers). Some of 
the teachers who responded to the 2011/12 Teacher Questionnaire were surveyed again one 
year later with the Questionnaire for Former Teachers or the Questionnaire for Current Teach- 
ers, which have different questions for teachers who are still teaching and for teachers who 
have stopped teaching. All public schools are included in the sampling frame; thus teachers 
can come from any public or public charter school. 

This study presents analyses of the responses from teachers who took the follow-up ques- 
tionnaires because those questionnaires asked about teachers’ satisfaction with the evalua- 
tion process. A total of 4,430 teachers were sampled as part of the Teacher Follow-up Survey, 
including 2,850 current teachers and 1,580 former teachers. The Schools and Staffing Survey 
and Teacher Follow-up Survey use a complex sampling design to produce a set of analytic 
weights that provide more accurate, unbiased estimates of a larger population. With these 
weights, the sample represents a population of more than 3 million teachers. See appendix B 
for more on the data sources. 

Methods 

The study team performed the following steps to conduct the analysis: 

• Teachers’ responses to relevant survey items were converted into numerical ratings of 
their perceptions of principal leadership and teacher influence. For the third, fourth, and 
fifth research questions, teachers’ ratings of their satisfaction with the evaluation process 
were collapsed from a four-point scale to a binary variable (translating a range of respons- 
es into satisfied or not satisfied). 

• Teachers’ survey responses were analyzed to produce estimated averages, frequencies, 
and percentages for all variables in the study (see appendix C). Tests of association, 
including chi-square tests of association and Pearson’s product moment correlations, 
were used to test for associations or correlations between variables. 

• Statistical analyses were conducted using logistic regression to predict the odds of a teacher 
being satisfied or not with his or her evaluation after school professional climate, teacher 
demographic characteristics, teacher professional characteristics, school characteristics, 
and whether the evaluation process included student test scores were controlled for. 

• For all analyses, sampling weights were used to ensure unbiased population estimates. 
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What the study found 


This section first describes the extent to which teachers were satisfied with the evaluation 
process and their perceptions of school professional climate. It then presents the results of 
analyses of the association between teachers’ satisfaction with the evaluation process and 
several other factors, including teachers’ perceptions of school professional climate, teacher 
demographic characteristics, teacher professional characteristics, school characteristics, 
and whether the evaluation process included student test scores. 


Most teachers reported being satisfied with the process by which they were evaluated 


Overall, 79 percent of teachers in the sample reported strongly agreeing or somewhat 
agreeing that they were satisfied with the evaluation process conducted in the previous 
year (figure 1). 

Teachers reported positive perceptions of their principal’s leadership but perceived themselves to 
have little influence over school policies 

On average, teachers somewhat agreed that their principal provides positive leadership, as 
measured by the composite variable for principal leadership (mean of 3.1 on a scale of 1, 
strongly disagree, to 4, strongly agree; table 1). Teachers on average reported having minor 
influence over school policies (mean of 2.2 on a scale of 1, no influence, to 4, a great deal 
of influence), such as setting performance standards for students, establishing curriculum, 
and hiring full-time teachers, among others. 


Teachers with 
more positive 
perceptions of 
their principal’s 
leadership were 
more likely to be 
satisfied with the 
evaluation process 


Of the two concepts used to measure school professional climate-principal leadership and teacher 
influence-only principal leadership was associated teachers’ satisfaction with the evaluation process 

Teachers with more positive perceptions of their principal’s leadership were more likely 
to be satisfied with the evaluation process. Teachers who rated their principal higher on 


Figure 1. Some 79 percent of teachers reported satisfaction with the evaluation 
process in 2011/12 



Note: n = 3,810 weighted to represent a population estimate of 2,846,477. Refers to responses to the 
question “I was satisfied with the formal evaluation process." Teachers were asked in 2012/13 to report their 
satisfaction with the evaluation process based on their experiences in 2011/12. 

Source: National Center for Education Statistics 2012/13 Teacher Follow-up Survey (Questionnaire for Former 
Teachers and Questionnaire for Current Teachers). 
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Table 1. Statistics for teachers' responses to survey items about school 


professional climate, 2011/12 

Area of school 
professional climate 

Mean 

Standard 

error 

Minimum 

value 

Maximum 
value \ 

Principal leadership 3 

3.1 

0.02 

1 

4 

Teacher influence 11 

2.2 

0.02 

1 

4 


Note: n = 4,430 weighted to represent a population estimate of 3,377,858. 

a. A composite variable indicating level of agreement (on a scale of 1, strongly disagree, to 4, strongly agree) 
with six items related to principal leadership (see appendix B). 

b. A composite variable indicating teachers’ perception of their influence (on a scale of 1, no influence, to 4, a 
great deal of influence) on seven items related to school policy (see appendix B). 

Source: Authors' calculations based on data from the National Center for Education Statistics 2011/12 
Schools and Staffing Survey (Teacher Questionnaire). 


leadership items tended to be more satisfied with the evaluation process. Specifically, 
the odds that a teacher was satisfied with the evaluation process increased 2.5 times for 
every one point increase in the rating of principal leadership, after teacher demographic 
characteristics, teacher professional characteristics, school characteristics, and whether the 
evaluation process included student test scores were controlled for (figure 2; see also table 
C12 in appendix C). This finding was statistically significant. 


Teachers’ 
perceptions of 
their influence 
over school 
policies were not 
associated with 
their satisfaction 
with the evaluation 
process 


Figure 2. Odds ratios of key variables show that teachers’ satisfaction with the 
evaluation process was related to principal leadership, 2011/12 

Principal leadership*** 


Teacher influence 

Satisfactory or effective evaluation rating 
(compared with excellent, outstanding, 
or highly effective evaluation rating)** 

Unsatisfactory or not that effective evaluation rating 
(compared with excellent, outstanding, 
or highly effective evaluation rating)*** 


Student test scores included in the evaluation process 
(compared with not included)*** 

0.0 0.5 1.0 1.5 2.0 2.5 

Odds ratio 

** is significant at p < .01; *** is significant at p < .001. 

Note: n = 3,080 weighted to represent a population estimate of 2,389,401. Teachers were asked in 2012/13 
to report their evaluation ratings from 2011/12 and to report their satisfaction with the evaluation process 
based on their experiences in 2011/12. An odds ratio that approaches 0 indicates that the outcome is ex- 
tremely unlikely to occur, an odds ratio of less than 1 indicates that the outcome is less likely, an odds ratio of 
1 indicates an equal likelihood of the outcome, and an odds ratio of more than 1 indicates that the outcome is 
more likely. For continuous variables, such as teacher influence, which has a scale of 1 to 4, the odds ratio is 
the change in the odds of being satisfied for every one unit increase in the continuous variable. Odds ratios for 
dichotomous variables (such as whether the evaluation process included student test scores) compared the 
likelihood of the two groups being satisfied with the evaluation process. An odds ratio of zero can be a statis- 
tically significant finding. The model overall is significant: F(20, 68) = 4.4, Prob > F = 0.00, design degrees of 
freedom = 87. Goodness-of-fit test: Hosmer & Lemeshow F(9,79) = 0.5, Prob > F = 0.9. 

Source: Authors’ calculations based on data from the National Center for Education Statistics 2011/12 Schools 
and Staffing Survey (School Questionnaire and Teacher Questionnaire) and 2012/13 Teacher Follow-up Survey 
(Questionnaire for Former Teachers and Questionnaire for Current Teachers). 
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Table 2. Teachers’ self-reported evaluation ratings from 2011/12, weighted estimates 

1 Rating 

Percent 

Standard error ■ 

Excellent, outstanding, or highly effective 

60.3 

1.7 

Satisfactory or effective 

36.8 

1.7 

Unsatisfactory or not that effective 

2.9 

0.7 


Note: Teachers were asked in 2012/13 to report their evaluation ratings from 2011/12. 

Source: Authors' calculations based on data from the National Center for Education Statistics 2012/13 
Teacher Follow-up Survey (Questionnaire for Former Teachers and Questionnaire for Current Teachers). 


But teachers’ perceptions of their influence over school policies were not associated with 
their satisfaction with the evaluation process. This finding was consistent before and after 
controlling for teacher demographic characteristics, teacher professional characteristics, 
school characteristics, and whether the evaluation process included student test scores (see 
table Cl 2 in appendix C). 

Teachers who were rated at the highest level on their evaluation in 2011/12 expressed more 
satisfaction with the evaluation process that year than teachers who were rated at lower levels 

Almost all teachers in the sample (97 percent) reported receiving a favorable rating in the pre- 
vious school year (2011/12). Specifically, 60.3 percent reported being rated excellent, outstand- 
ing, or highly effective, and 36.8 percent reported being rated satisfactory or effective (table 2). 

The only teacher professional characteristic significantly associated with satisfaction with 
the evaluation process was teachers’ self-reported previous-year evaluation rating (figure 2; 
see also table C12 in appendix C). Specifically, teachers who received an evaluation rating 
of satisfactory or effective were 0.4 times less likely to be satisfied with the evaluation process 
than those who received an evaluation rating of excellent, outstanding, or highly effective. 


The only teacher 
professional 
characteristic 
significantly 
associated with 
satisfaction with 
the evaluation 
process was 
teachers’ 
self-reported 
previous-year 
evaluation rating 


No other teacher professional characteristics (school level taught, years of teaching experi- 
ence, and teaching status) and no teacher demographic characteristics (gender, ethnicity, 
and race) or school characteristics (enrollment, urbanicity, percentage of students who are 
English learner students, percentage of students approved for free or reduced-priced lunch, 
percentage of students with an individualized education program, and percentage of stu- 
dents who are a racial/ethnic minority) were associated with teachers’ satisfaction with the 
evaluation process. 


Teachers whose evaluation process included student test scores were less likely to be satisfied with 
the evaluation process than teachers whose evaluation process did not include student test scores 


Teachers whose evaluation process included student test scores were 2.5 times less likely 
(odds ratio of 0.4) to be satisfied with the evaluation process than teachers whose evalu- 
ation process did not (see figure 2 and table C12 in appendix C). This finding remained 
consistent after teacher demographic characteristics (gender, ethnicity, and race), teacher 
professional characteristics (school level taught, years of teaching experience, previous-year 
evaluation rating, and teaching status), and school characteristics (enrollment, urbanicity, 
percentage of students who are English learner students, percentage of students approved 
for free or reduced-price lunch, percentage of students with an individualized education 
program, and percentage of students who are a racial/ethnic minority) were controlled for. 
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Because the framework for evaluation systems is usually determined at the state level, it 
is possible that the significant relationship between the inclusion of student test scores in 
evaluations and teachers’ satisfaction with the evaluation process may actually be captur- 
ing differences across states. However, the relationship remained significant when tested in 
an alternative model that controlled for differences across state teacher evaluation policies 
(see appendix B for details and table C13 in appendix C). Approximately 27 percent of 
teachers reported that the evaluation process included student test scores, such as growth 
models or value-added models, in 2011/12 (see table C3 in appendix C). This study does 
not address whether the relationship of lower satisfaction to the inclusion of student test 
scores is driven fully or partially by the new evaluation systems themselves or by the length 
of time the systems have been in place. Almost all states were revising their teacher eval- 
uation policies during the time period covered by this study. Indeed, 30 of them required 
the inclusion of some measure of student learning by 2012 (National Council on Teacher 
Quality, 2012), although at the time of the study, only one state, Florida, was fully imple- 
menting its new evaluation system. 

Implications of the study findings 


The findings from this study present several implications for future research, policy, and 
school practice. 

The finding that principal leadership — as one measure of school professional climate — 
was associated with teachers’ satisfaction with the evaluation process reinforces previous 
research about the importance of principals in establishing a positive school profession- 
al climate (Drago-Severson, 2012; Grissom, 2011; Kraft, Marinell, & Yee, 2015). It also 
reinforces research on Chicago’s new teacher evaluation program, which similarly found 
that teachers’ perceptions of their evaluation were related to both school leadership and 
the concepts of professional community (such as principal-teacher trust; Jiang, Sporte, & 
Luppescu, 2015). This finding might be because teachers view evaluation as a “top-down” 
activity, requiring the principal or other supervisor to render an assessment of their work, 
or it might be related to the principal’s role of translating and implementing school policy. 
Further research could examine the following questions: 

• Is there a causal relationship between principal leadership and teachers’ sat- 
isfaction with the evaluation process? What is the nature and direction of that 
relationship? 

• What elements of leadership — such as establishing a culture of trust or providing 
appropriate supports to teachers — are most directly related to teachers’ satisfaction 
with the evaluation process? 

States in the Northeast Educator Effectiveness Research Alliance might be interested in 
further exploring the relationships between school professional climate and teachers’ sat- 
isfaction with the evaluation process. For example, Maine and Vermont, which are just 
beginning to implement new evaluation systems, might gather data before and after imple- 
mentation to address this issue. State leaders may be interested in investigating whether 
and to what extent school professional climate is related to how successfully the new eval- 
uation processes are implemented. 

Other questions include: 


The finding 
that principal 
leadership was 
associated 
with teachers’ 
satisfaction with 
the evaluation 
process reinforces 
previous 
research about 
the importance 
of principals in 
establishing a 
positive school 
professional 
climate 
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• Are there different relationships between principal leadership and teachers’ satis- 
faction with the evaluation processes in different Northeast Educator Effectiveness 
Research Alliance states? Are the differences related to key policy characteristics 
in the state? 

• To what extent does successful implementation of a new system vary by teachers’ 
perceptions of the new systems? 

When the National Center for Education Statistics 2011/12 Schools and Staffing Survey 
and 2012/13 Teacher Follow-up Survey were administered, teachers’ evaluation ratings 
varied little, and teachers’ satisfaction with the evaluation process was high. Teachers 
overwhelmingly reported being rated effective or highly effective, with 60.3 percent rated 
excellent, outstanding, or highly effective and 36.8 percent rated as satisfactory or effective. 
Indeed, lack of differentiation in teacher ratings has been a criticism of and motivator for 
new evaluation systems (Toch & Rothman, 2008; Weisberg, Sexton, Mulhern, & Keeling, 
2009). At the time of the study, many states were still revamping their educator evaluation 
systems, and some had begun implementation. Now that a majority of states have fully 
implemented new evaluation systems, it would be informative to replicate the study to see 
whether the results are consistent. 

The finding that teachers are less likely to be satisfied with the evaluation process when 
student test scores are included warrants further exploration, given that many policy- 
makers advocate for evaluation systems that include a measure of student achievement or 
growth. In the REL Northeast & Islands Region the weight of test scores in new evalua- 
tion systems varies across states, providing an opportunity to understand whether and how 
different weights of test scores in evaluation are related to teachers’ satisfaction. However, 
the finding might be because teachers believe that student test scores do not accurately 
measure their instructional practice (Coggshall, Ott, & Lasagna, 2010; Darling-Hammond, 
2015). Further research on this topic could examine the following questions: 

• Does satisfaction depend on specific features, such as the weight of test scores or 
the types of student learning measures? 

• Does satisfaction with the teacher evaluation system depend on the amount of 
time the system has been in place? 

• Does satisfaction with the teacher evaluation system depend on the subject area 
taught (for example, math, music, or special education)? 

Limitations of the study 


The finding that 
teachers are 
less likely to be 
satisfied with the 
evaluation process 
when student 
test scores are 
included warrants 
further exploration, 
given that many 
policymakers 
advocate for 
evaluation systems 
that include a 
measure of student 
achievement 
or growth 


This study has four main limitations. 

First, the study finds an association between teachers’ satisfaction with the evaluation 
process and their perceptions of school professional climate (that is, principal leadership 
and teacher influence) but does not support causal conclusions about the association. 

Second, the findings are from a national sample, but the education systems and evaluation 
policy contexts of individual states may differ from the average or representative school in 
the National Center for Education Statistics Schools and Staffing Survey dataset. 

Third, the study is a national snapshot of the relationship between school profession- 
al climate and teacher evaluation processes at a particular time. The relationships may 
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change as evaluation policies shift. At the time of the survey many states were imple- 
menting new teacher evaluation systems, but it was not possible to use the Schools and 
Staffing Survey and Teacher Follow-up Survey to determine whether teachers’ satisfaction 
was related to the stage of implementation of the new evaluation systems. 

Fourth, the Schools and Staffing Survey measures only two dimensions of school profes- 
sional climate (principal leadership and teacher influence on school policy). The research 
suggests that other dimensions, such as trust, that were not considered in this study may 
be related to support of or satisfaction with the evaluation process (Riordan et al, 2015). 
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Appendix A. Literature review 


This study draws on research and policy literature on teacher evaluation reform, school 
professional climate, and the inclusion of student outcomes in teacher evaluations. 

Teacher evaluation reform 

Recent research has identified the need to reform teacher evaluation systems because many 
systems are considered pro-forma, lack rigor, are unable to differentiate teacher quality, and 
have little impact on or relationship to student outcomes (Daley & Kim, 2010; Gordon, 
Kane, & Staiger, 2006; Heneman, Milanowski, Kimball, & Odden, 2006; Measures of 
Effective Teaching Project, 2010, 2012; Toch & Rothman, 2008; Weisberg et al., 2009). 

School professional climate 

Researchers have examined the role of school professional climate — also known as school 
culture or teachers’ working conditions — on principals, teachers, and students. Common 
components across frameworks that define school professional climate include structural 
conditions, such as facilities and resources specific to instruction; relational conditions, 
such as opportunities to collaborate and form community; and emotional or psychological 
conditions, such as safety, trust, and self-efficacy (Cohen, McCabe, Michelli, & Pickeral, 
2009; Drago-Severson, 2012; Johnson, 2006). 

Overall, the literature supports the idea that schools’ professional climate contributes 
to teachers’ job satisfaction, which is ultimately related to teacher retention or turnover 
(Borman & Dowling, 2008) and student achievement (Johnson, Kraft, & Papay, 2012; 
Ladd, 2011). Working conditions, in addition to salaries and benefits, figure highly in 
teachers’ career plans as they consider whether to stay at their current school (Boyd et al., 
2011; Ladd, 2011). Johnson, Papay, Liarman, Munger, and Qazilbash (2010), who analyzed 
evidence from a Massachusetts statewide school survey on teachers’ working conditions 
along with school and student achievement data, found that working conditions matter as 
predictors of teachers’ satisfaction and career intentions: as working conditions improve, 
teachers’ intentions to transfer tend to decrease. 

School professional climate is also associated with teacher effectiveness. Researchers using 
data from North Carolina’s Charlotte-Mecklenburg Schools found that teachers working 
in more supportive professional environments improved their effectiveness (as measured by 
student achievement on standardized tests) more over time than teachers working in less 
supportive contexts (Kraft & Papay, 2014). Another study found that schools with overall 
better working conditions also tended to have higher student achievement in math and 
English language arts (Johnson et al, 2012). 

Studies have found that the school principal plays an important role in establishing and 
maintaining positive working conditions. Drago-Severson’s (2012) study of 25 principals in 
public, private, and religious schools found that principals understood their role in shaping 
professional school climate to support teachers and employed strategies to strengthen the 
professional climate, such as promoting shared values, sharing leadership with teachers, and 
supporting teacher collaboration. Moreover, teachers’ positive perceptions of their princi- 
pals might lead them to remain in their schools. Grissom’s (2011) study using data from 
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the 2003/04 Schools and Staffing Survey and the 2004/05 Teacher Follow-up Survey found 
that teachers who rated their principal as more effective were more satisfied with their jobs 
and less likely to leave within a year. Kraft et al.’s (2015) study found that improvements in 
school leadership were associated with reductions in teacher turnover. 

Inclusion of student outcomes in teacher evaluations 

While researchers and policymakers debate the value and reliability of including student 
outcomes and measures of student growth in teacher evaluations (for example, Darling- 
Hammond, 2012), states and districts have begun incorporating them. A National Council 
on Teacher Quality (2012) report found that as of 2012, 30 states require that teacher 
evaluations include evidence of student learning, 25 states require that evaluation systems 
differentiate ratings into more than two categories, and 39 states require annual class- 
room observations. The same report noted that teacher evaluations have higher stakes, 
with nine states mandating that teacher tenure decisions be tied to student performance 
(National Council on Teacher Quality, 2012). 

Emerging research examines the use of student outcomes in teacher evaluations. Denver’s 
evaluation of the ProComp System pilot, in which teachers could voluntarily opt into a 
new evaluation system with differentiated compensation, found that students experienced 
gains regardless of whether their teacher participated in the new system (Goldhaber & 
Walch, 2011). Many studies have focused on the relationship between measures of student 
learning, such as growth or value-added scores, and other evaluation measures, such as 
observations of teachers. In a study of evaluation instruments used in the pilot of Chicago 
Public Schools’ new evaluation system, the Chicago Consortium for School Research 
found that observation ratings (including ratings of the same teacher by more than one 
observer) correlated with student performance on achievement tests (Sartain, Stoelinga, 
& Brown, 2011). Similarly, the Measures of Effective Teaching project found that teach- 
ers with value-added scores also perform well on other measures, such as on observation 
rubrics and ratings from their students (Measures of Effective Teaching Project, 2010, 
2012). The findings have not been consistent, however. Walsh and Lipscomb’s (2013) com- 
parison of observation ratings by principals and evaluators in Pennsylvania’s evaluation 
system pilot with value-added scores for the same teachers found that observation scores 
clustered in the two high categories while the value-added estimates were more variable. 
Daley and Kim’s (2010) study of the System for Teacher and Student Advancement found 
a correlation between observational components and value-added components in teacher 
evaluation scores. 
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Appendix B. Data sources and study methods 


This appendix describes the data sources and study methods. 

Data sources 

Data are from the National Center for Education Statistics (NCES) Schools and Staffing 
Survey and Teacher Follow-up Survey, which collect information on U.S. elementary and 
secondary schools and their staffs through surveys of districts, schools, principals, teachers, 
and library media centers (Goldring, Taie, Rizzo, Colby, & Fraser, 2013). This study relied 
on data from four linked survey questionnaires: 

• The 2011/12 Schools and Staffing Survey School Questionnaire. 

• The 2011/12 Schools and Staffing Survey Teacher Questionnaire. 

• The 2012/13 Teacher Follow-up Survey Questionnaire for Former Teachers. 

• The 2012/13 Teacher Follow-up Survey Questionnaire for Current Teachers. 

Some of the teachers who responded to the 2011/12 Teacher Questionnaire were surveyed 
again one year later with a follow-up questionnaire that has different questions for teach- 
ers who are still teaching and for teachers who have stopped teaching. The dataset links 
teachers’ responses to the Teacher Follow-up Survey with their responses to the previous 
year’s Schools and Staffing Survey Teacher Questionnaire and to their school’s responses 
to the previous year’s Schools and Staffing Survey School Questionnaire. The sample of 
interest for this study was current and former teachers who were asked in 2012/13 about 
evaluations conducted the previous school year (table Bl). 

Schools and Staffing Survey and Teacher Follow-up Survey procedures for sampling and 
weighting ensure that the samples of schools and teachers are representative at the nation- 
al, regional, and state levels. The survey datasets contain a series of weights that, when 
used with the appropriate statistical software procedures, increase the accuracy of esti- 
mates by using adjusted standard errors for all statistics, such as means and frequencies. 
Standard errors are adjusted using balanced repeated replication procedures to account for 
nonresponse and for the complex, stratified design of the samples. 1 

The variables of interest to this study, their purpose, source, and relevant details, are 
shown in table B2. The key outcome variable is teachers’ satisfaction with the evaluation 
process. The Schools and Staffing Survey asks this as: “To what extent do you agree or 
disagree with each of the following statements about the formal evaluation of your work as 
a teacher last school year (2011/12): I was satisfied with the formal evaluation process.” The 
answer options are a four-point scale of strongly disagree, somewhat disagree, somewhat 


Table Bl. Teaching status in Teacher Follow-up Survey sample, 2012/13 


Teaching 

Unweighted 

Weighted estimated 

Standard 

status 

percentage 

percentage 

error 

Former teachers 

35.6 

7.7 

0.6 

Current teachers 

64.4 

92.3 

0.6 


Note: n = 4,430 unweighted to represent a population of almost 3 million teachers. 

Source: Authors' calculations based on data from the National Center for Education Statistics 2012/13 
Teacher Follow-up Survey (Questionnaire for Former Teachers and Questionnaire for Current Teachers). 


B-l 




agree, and strongly agree. To conduct the logistic regression to assess whether school pro- 
fessional climate is associated with teachers’ satisfaction with the evaluation process, the 
satisfaction variable was transformed into a dichotomous variable. Strongly agree and 
somewhat agree were grouped into an agree category. Strongly disagree and somewhat dis- 
agree were grouped into a disagree category. 

Schools and Staffing Survey and Teacher Follow-up Survey data are collected by the 
Census Bureau for the NCES (Goldring et al., 2013). The data collection period for the 
2011/12 School Questionnaire and Teacher Questionnaire was from June 2011 to June 
2012. The data collection period for the 2012/13 Questionnaire for Former Teachers and 
Questionnaire for Current Teachers was January 2013 to August 2013. Teachers could 
respond to all surveys via a web-based questionnaire or on paper. 


Table B2. Schools and Staffing Survey and Teacher Follow-up Survey variables used in this study, 
2011/12 and 2012/13 

Variable 

Question 

Purpose 

Survey source 

Details 

Explanatory variable 

Satisfaction 

with evaluation 

process 

To what extent do you agree or disagree 
with each of the following statements 
about the formal evaluation of your work 
as a teacher last school year (2011/12): 

1 was satisfied with the formal evaluation 

process? 

Dependent 

variable 

2012/13 Teacher 
Follow-up Survey 
(Questionnaire for 

Former Teachers and 

Questionnaire for 
Current Teachers): 

F1323 

• Categorical variable: 
strongly agree, somewhat 
agree, somewhat disagree, 
strongly disagree 

• Collapsed to two 
categories (strongly 
agree/somewhat agree 
and somewhat disagree/ 
strongly disagree) for the 
third, fourth, and fifth 
research questions 

Independent variables 

Principal 

leadership 

To what extent do you agree or disagree 
with each of the following statements? 

Key 

independent 

variable 

2011/12 Schools 
and Staffing 

Survey (Teacher 
Questionnaire): 
composite of T0435, 
T0441, T0446, 

T0449, T0451, and 

T0467 

• Composite of six items 
selected via principal 
components analysis 

• Continuous variable on a 

scale of 1-4 

Teacher 

influence 

How much influence do you think teachers 
have over school policy AT THIS SCHOOL in 
the following areas? 

Key 

independent 

variable 

2011/12 Schools 
and Staffing 

Survey (Teacher 
Questionnaire): 
composite of T0420, 
T0421, T0422, 

T0423, T0424, 

T0425, and T0426 

• Composite of seven items 

• Continuous variable on a 

scale of 1-4 

Evaluation 

process includes 

student test 

scores 

Are student test score outcomes or test 

score growth included as an evaluation 
criterion in your FORMAL evaluation this 
school year? 

Key 

independent 

variable 

2011/12 Schools 
and Staffing 

Survey (Teacher 
Questionnaire): 

T0403 

• Dichotomous variable: yes 

or no 

(continued) 
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Table B2. Schools and Staffing Survey and Teacher Follow-up Survey variables used in this study, 
2011/12 and 2012/13 (continued) 

Variable 

Question 

Purpose 

Survey source 

Details 

Teacher demographic characteristics 

Gender 

Are you male or female? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (Teacher 

Questionnaire): 

GENDER_S 

■ Dichotomous variable: 

female or male 

Ethnicity 

Are you of Hispanic or Latino origin? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (Teacher 
Questionnaire): 

T0527 

• Dichotomous variable: 

Hispanic or not Hispanic 

Race 

What is your race? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (Teacher 
Questionnaire): RACE 

• Categorical variable 

• Collapsed to three 
categories: White, Black, 

or other 

Teacher professional characteristics 

School level 
taught 

Do you currently teach students in any of 
these grades AT THIS SCHOOL? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (Teacher 
Questionnaire): 

TLEVEL 

• Dichotomous variable: 
elementary or secondary 

Years of 

teaching 

experience 

How many school years have you worked 
as an elementary- or secondary-level 
teacher in public, public charter, or private 

schools? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (Teacher 

Questionnaire): 

TOTYREXP_S 

• Continuous variable 

Teaching status 

Do you CURRENTLY TEACH any regularly 
scheduled class(es) in any of grades 
preK-12? 

Other 

covariate 

2012/13 Teacher 
Follow-up Survey 
(Questionnaire for 

Former Teachers and 

Questionnaire for 
Current Teachers): 
STTUSJT 

■ Dichotomous variable: 

current teacher or former 

teacher 

Previous-year 
evaluation rating 

Which of the following best describes 
the evaluation rating you received for 
your work as a teacher last school year 
(2011/12)? 

Other 

covariate 

2012/13 Teacher 
Follow-up Survey 
(Questionnaire for 

Former Teachers and 

Questionnaire for 
Current Teachers): 
STTUSJT 

• Categorical variable: 
excellent, outstanding, 
or highly effective; 
satisfactory or effective; or 
unsatisfactory or not that 

effective 

School characteristics 

Enrollment 

Around the first of October, how many 
students in grades K-12 and comparable 
ungraded levels were enrolled in this 

school? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (School 
Questionnaire): 

ENRK12 

■ Continuous variable 

Urbanicity 

Based on census location of school 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (School 
Questionnaire): 

URBANS12 

• Categorical variable 

• Collapsed from 12 
categories to 4: city, 
suburb, town, rural 

(continued) 
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Table B2. Schools and Staffing Survey and Teacher Follow-up Survey variables used in this study, 
2011/12 and 2012/13 (continued) 

Variable 

Question 

Purpose 

Survey source 

Details 

Percentage of 

students who 

are English 

learner students 

How many limited-English proficient 
students or English-language learners are 

enrolled in this school? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (School 
Questionnaire): LEP 

• Continuous variable 

• Calculated as a percentage 

Percentage 

of students 

approved for free 
or reduced-price 
lunch 

Around the first of October, how many 

students at this school were APPROVED 

for free or reduced-price lunches? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (School 
Questionnaire): 
NSLAPP_S 

• Continuous variable 

• Calculated as a percentage 

Percentage of 
students with 

an individualized 

education 

program 

How many students have an Individual 
Education Plan (IEP) because they have 
special needs? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (School 
Questionnaire): IEP 

• Continuous variable 

• Calculated as a percentage 

Percentage of 

students who 

are a racial/ 
ethnic minority 

Around the first of October, how many 
students enrolled in grades K-12 and 
comparable ungraded levels were: 

• Hispanic or Latino regardless of race? 

• White, not of Hispanic or Latino origin? 

• Black or African American, not of 

Hispanic or Latino origin? 

• Asian, not of Hispanic or Latino origin? 

■ Native Hawaiian or other Pacific 

Islander, not of Hispanic or Latino 
origin? 

• American Indian or Alaska Native, not of 
Hispanic or Latino origin? 

Two or more races, not of Hispanic or 
Latino origin? 

Other 

covariate 

2011/12 Schools 
and Staffing 

Survey (School 
Questionnaire): 
MINENR 

• Continuous variable 

• Calculated as a percentage 

Sample definition variables 

Evaluated in 

2011/12 

Were you, or are you going to be, rated in a 
FORMAL evaluation this school year? 

Sample 

definition 

2011/12 Schools 
and Staffing 

Survey (Teacher 
Questionnaire): 

T0402 

• Dichotomous: yes or no 

• Only those responding yes 

are included 

Evaluated last 

school year 
(2011/12) 

Were you formally evaluated for your work 
as a teacher last school year (2011/12)? 

Sample 

definition 

2012/13 Teacher 
Follow-up Survey 
(Questionnaire for 

Former Teachers and 

Questionnaire for 
Current Teachers): 

F1320 

• Dichotomous: yes or no 

• Only those responding yes 
are included 

Source: National Center for Education Statistics 2011/12 Schools and Staffing Survey (School Questionnaire and Teacher Question- 
naire) and 2012/13 Teacher Follow-up Survey (Questionnaire for Former Teachers and Questionnaire for Current Teachers). 
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Sampling 


Teachers sampled for the Teacher Follow-up Survey were drawn from the Schools and 
Staffing Survey teacher sample, which was drawn from the Schools and Staffing Survey 
school sample. The Schools and Staffing Survey defines teachers as staff members who 
teach regularly scheduled classes to students in any K-12 grade. Because the Schools and 
Staffing Survey and the Teacher Follow-up Survey are so interrelated, the description of 
sampling frames and sample selection begins with the Schools and Staffing Survey and 
then moves to the Teacher Follow-up Survey sampling process. 

2011/12 Schools and Staffing Survey. The 2011/12 Schools and Staffing Survey universe 
includes schools in the 50 states plus the District of Columbia and excludes other jurisdic- 
tions, Department of Defense overseas schools, Bureau of Indian Education schools, and 
Common Core of Data schools that do not offer teacher-provided classroom instruction in 
grades 1-12 or the ungraded equivalent. Lists of teachers for each sampled school are gen- 
erated, and a sample of teachers from the lists is selected. Sampling is conducted so that 
results of national, regional, and state estimates of all variables can be reliably produced. 

The maximum number of teachers per school was set at 20 to reduce the burden on 
schools. About 18 percent of the eligible public schools and 23 percent of the eligible 
private schools did not provide teacher lists, so no teachers were selected for those schools. 
About 51,100 public school teachers and 7,100 private school teachers were sampled. Full 
details of the sampling plan for the surveys can be found in Goldring et al. (2013). 

2012/13 Teacher Follow-up Survey. The sampling frame for the Teacher Follow-up 
Survey consisted of the public and private school teachers who completed the Schools 
and Staffing Survey 2011/12 Teacher Questionnaire. The sample for the Teacher Follow-up 
Survey is a stratified sample allocated to allow comparisons of teachers by teaching status 
(stayers, movers, and leavers) within sector (traditional public, public charter, and private), 
experience groups (first year, second year, third year, fourth or fifth year, and experienced), 
grade level taught (primary, middle, or high school), and teacher’s race/ethnicity (White, 
Hispanic, Black, or other). Information collected from schools at the beginning of the 
2012/13 school year was used to determine a teacher’s status as stayer, mover, leaver, or 
unknown. More details about the sampling for the Teacher Follow-up Survey can be found 
in Goldring, Taie, & Riddles (forthcoming). 

For this study, only regular teachers who teach grade K-12 or a comparable ungraded level 
in any public school and who responded to the Teacher Follow-up Survey were included 
because these teachers were more likely to be covered by evaluation systems. 

Response rates and bias analysis 

NCES data cleaning procedures include checks for response bias, such that if the unit or 
item response rates fall below certain thresholds, the data files are examined for possible 
bias. The nonresponse bias analysis checks district, school, and teacher characteristics for 
evidence of bias. 

NCES considers two types of nonresponse for the Schools and Staffing Survey: unit-level 
nonresponse (that is, a questionnaire was not filled out or only partially filled out by an 
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eligible respondent, and key questionnaire items were left out) and itemdevel nonresponse 
(that is, questionnaire items were missing in a questionnaire that had passed the threshold 
to be considered completed). 

For the 2011/12 Schools and Staffing Survey the weighted unit response rate was 
72.5 percent on the School Questionnaire (for public schools only) and 77.7 percent on the 
Teacher Questionnaire (for public school teachers only). The item response rates for the 
School Questionnaire and the Teacher Questionnaire were very high: over 94 percent of 
items on the Teacher Questionnaire and the School Questionnaire had a response rate of 
85 percent or higher. Between 4 percent and 6 percent of the items on both questionnaires 
had a response rate of 70.0-84.9 percent. NCES analyzed these items for potential bias. 

The 2012/13 Teacher Follow-up Survey underwent similar nonresponse analyses by NCES. 
The weighted unit response rate was 73.8 percent for the Questionnaire for Former Teach- 
ers and 81.3 percent for the Questionnaire for Current Teachers. The item response rate 
was moderate: the percentage of items with a response rate of 85 percent or higher was 
72 percent for the Questionnaire for Former Teachers and 84 percent for the Question- 
naire for Current Teachers. Both questionnaires contained items with response rates below 
70 percent, which is considered a low response rate according to NCES. Items with low 
response rates were not used in this study. 

The study team conducted nonresponse analyses for each variable used in the analysis 
(table B2). Overall, rates of nonresponse were low, less than 2 percent. Two variables — 
satisfaction with the evaluation process and the previous-year evaluation rating — had an 
item nonresponse rate of about 9 percent. 

The Schools and Staffing Survey datasets provided by NCES contain imputed data for 
items with missing data. The results shown in the main report were estimated using 
imputed data. The logistic regression results were rerun without imputed data, and the 
results were consistent. 

To measure the key variables of interest related to school professional climate, the study 
team built on a study of the implementation of educator evaluations in New Hampshire 
(Riordan et al., 2015), which identified three constructs of school professional climate 
related to teachers’ support of the new evaluation system. The three constructs are prin- 
cipal leadership, trust, and teacher influence. The Schools and Staffing Survey includes 
items related to two of these constructs of school professional climate — principal leader- 
ship and teacher influence — but does not contain items related to teacher trust. 

Both professional climate variables were constructed by making a composite based on 
teacher responses to a series of questions about principal leadership and teacher influence. 

The principal leadership composite was created by averaging each respondent’s response 
to six items that address leadership issues in the school. These six items came from two 
questions that ask about a wide range of items related to school climate. These items were 
selected because they relate to or were specifically about the leadership and climate or 
were similar to items included in previous research about professional climate and imple- 
mentation of new evaluation systems (Riordan et al., 2015). The study team selected the 
six items after examining a set of 12 items in a principal components analysis. The results 
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of the principal components analysis suggested the items were measuring two different 
underlying concepts. Six items were strongly related to principal leadership. The other 
items addressed teacher collaboration and were thus removed from the principal leadership 
composite. Cronbach’s alpha was calculated on the six items identified for the leadership 
composite and showed high internal consistency (Cronbach’s alpha = .86). The six items 
in the leadership composite are: 

• The school administration’s behavior toward the staff is supportive and 
encouraging. 

• The principal enforces rules for student conduct and backs me up when I need it. 

• The principal knows what kind of school he or she wants and has communicated 
it to the staff. 

• In this school, staff members are recognized for a job well done. 

• I am generally satisfied with being a teacher at this school. 

• I like the way things are run at this school. 

The items are similarly phrased and have the identical response option scale of strong- 
ly agree, somewhat agree, somewhat disagree, and strongly disagree. The scale originally 
indicated that strongly agree carried a value of 1, so the response options were reverse 
coded such that the more positive response of strongly agree would be 4- The principal 
leadership composite score for any individual respondent ranged from 1 to 4, with a higher 
value indicating a more positive teacher perception of school leadership. The mean value 
for leadership was 3.13. 

To construct the teacher influence composite, a principal components analysis was conduct- 
ed on seven items in a question that asks teachers “how much actual influence do you think 
teachers here have over school policy AT THIS SCHOOL in each of the following areas.” 
The seven items are similarly phrased and have the following identical response option scale: 
no influence, minor influence, moderate influence, and a great deal of influence, with affil- 
iated scores of 1 for no influence to 4 for a great deal of influence. The results of the princi- 
pal components analysis showed that the items were highly related to the same underlying 
concept, and Cronbach’s alpha was calculated to see whether the items had high internal 
consistency (Cronbach’s alpha = .8). The seven items included in the composite are: 

• Setting performance standards for students at this school. 

• Establishing curriculum. 

• Determining the content of in-service professional development programs. 

• Evaluating teachers. 

• Hiring new full-time teachers. 

• Setting discipline policy. 

• Deciding how the school budget will be spent. 

The composite variable “teacher influence” was calculated by averaging each respondent’s 
answers to the seven items above. The values of the new variable range from 1 to 4, with 
a higher value indicating more influence. The mean value of the teacher influence is 2.15. 

Data analysis 

For data analyses the study team used the NCES-recommended balanced repeated repli- 
cation procedures for weighting, which weights each response to produce more accurate 
estimates by compensating for sampling stratification and missing data. 
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Data were first analyzed descriptively, producing estimated means, frequencies, and per- 
centages, as appropriate, for each variable of interest (see appendix C). The study team 
tested for correlation between the explanatory and control variables and calculated pair- 
wise correlations of each variable, with the specific correlation measure used dependent on 
the type of variable. The chi-square test of independence was used to test the correlation 
between two categorical variables (for example, gender and race/ethnicity). Pearson’s r was 
used to test the correlation between two continuous variables (for example, school size and 
percentage of students approved for free or reduced-priced lunch). Parametric or nonpara- 
metric one-way analysis of variance was used to test the correlation between a continuous 
and categorical variable (for example, years of teaching and grade level taught). There is 
no absolute guidance on what level of correlation is too high to include both variables in 
the same regression. According to Hamilton (1990), correlations of ± .5 are moderate and 
those of ± .8 are strong. Variables with moderately high correlations, of ± .65 or greater, 
were excluded from the same regression equation. Examples of variables with high correla- 
tions where only one of the pair was included in the analyses are teacher’s age and years of 
teaching experience. 

To answer the first two research questions, the study team calculated frequencies and 
means for the variables of interest: teacher satisfaction with the evaluation process and 
measures of school professional climate. To answer the third, fourth, and fifth research 
questions, the study team conducted logistic regression, which predicts the odds of a 
teacher being satisfied with his or her evaluation given a set of predictors, such as teacher 
demographic characteristics, teacher professional characteristics, school characteristics, 
and whether the evaluation process included student test scores. The logistic regression 
used the set of Schools and Staffing Survey replicate weights. The model included the 
variables described in table C12 in appendix C. The explanatory or outcome variable was 
satisfaction with the evaluation process. The key independent variables were two indi- 
cators of school professional climate: principal leadership and teacher influence. A set of 
other variables of interest related to teacher demographic characteristics (gender, ethnic- 
ity, and race), teacher professional characteristics (school level taught, years of teaching 
experience, self-reported previous-year evaluation rating, and teaching status), and school 
characteristics (enrollment, urbanicity, percentage of students who are English learner stu- 
dents, percentage of students approved for free or reduced-priced lunch, percentage of stu- 
dents with an individual education program, and percentage of students who are a racial/ 
ethnic minority) were used. The final key variable of interest was whether the evaluation 
process included student test scores. Logistic regression results are presented in table C12 
in appendix C. The logistic regression model can be summarized as: 

( [1 - D]) = + + ^ 2 ^ 2i + ^ 3 ^ 3i + £ ' 

where D ; is a dummy variable for which 1 = satisfied with the evaluation and 0 = not sat- 
isfied, (3 0 is a constant term, _ k are the coefficients for the explanatory and control vari- 
ables, and X 1; _ K; are the values of the explanatory and control variables for each variable 
1-k and each observation. 

An odds ratio that approaches 0 indicates that the outcome is extremely unlikely, an odds 
ratio of less than 1 indicates that the outcome is less likely, an odds ratio of 1 indicates 
an equal likelihood of the outcome, and an odds ratio of more than 1 indicates that the 
outcome is more likely. An odds ratio of 0 can be a statistically significant finding. 
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Alternative model specification 


Decisions about the framework for the evaluation system, including whether the evalua- 
tion process included student test scores, are frequently made at the state level. Thus it is 
possible that the observed effect for this variable may be partially explained by the state in 
which the teacher works. To test this possibility, it would be ideal to run a logistic regres- 
sion with state fixed effects or a conditional logit model. However, it was not possible to 
run a logistic regression with both state fixed effects and teacher-specific replicate weights; 
only statedevel replicate weights are allowed. Thus, an alternative specification was run 
treating the outcome variable as continuous, including state fixed effects, and using rep- 
licate weights. The results of the alternative specification to the full model were almost 
identical, with odds ratios being consistent in size, direction, and significance. The results 
of this model are in table C13 in appendix C. 
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Appendix C. Supplementary tables and analyses 


This appendix provides descriptive statistics for all variables used in the analysis (tables 
Cl-Cll). Logistic regression results are provided in table C12, and an alternative specifica- 
tion of the model with state fixed effects is provided in table C13. 

Explanatory variable 


Table Cl. Teachers’ satisfaction with the evaluation process in 2011/12, weighted 


estimates 

Satisfied with evaluation process 

Percent 

Standard error j 

Somewhat agree or strongly agree 

78.9 

1.3 

Somewhat disagree or strongly disagree 

21.1 

1.3 


Note: Teachers were asked in 2012/13 to report their satisfaction with the evaluation process based on their 
experiences in 2011/12. 

Source: Authors' calculations using data from the National Center for Education Statistics 2012/13 Teacher 
Follow-up Survey (Questionnaire for Former Teachers and Questionnaire for Current Teachers). 


Independent variables 


Table C2. Teachers’ perceptions of school professional climate, weighted 
estimates, 2011/12 

1 School climate item 

Mean Standard error I 

Principal leadership 3 

3.1 0.02 

Teacher influence 11 

2.2 0.02 


a. A composite variable indicating level of agreement (on a scale of 1, strongly disagree, to 4 strongly agree) 
with six items related to principal leadership (see appendix B). 

b. A composite variable indicating teachers’ perception of influence (on a scale of 1, no influence, to 4, a great 
deal of influence) on seven items related to school policy (see appendix B). 

Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools 
and Staffing Survey (Teacher Questionnaire). 


Table C3. Evaluation process included student test scores, weighted estimates, 
2011/12 


Evaluation process included 
student test scores 

Percent 

Standard error 

Yes 

26.7 

1.2 

No 

73.3 

1.2 


Source: Authors' calculations using data from the National Center for Education Statistics 2012/13 Teacher 
Follow-up Survey (Questionnaire for Former Teachers and Questionnaire for Current Teachers). 
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Teacher demographic characteristics 


Table C4. Teachers’ gender, weighted estimates, 2011/12 


1 Gender 

Percent 

Standard error 1 

Female 

23.7 

0.5 

Male 

76.3 

0.5 

Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools 


and Staffing Survey (Teacher Questionnaire). 


Table C5. Teachers' ethnicity, weighted estimates, 2011/12 


1 Ethnicity 

Percent 

Standard error j 

Hispanic 

7.7 

0.4 

Not Hispanic 

92.3 

0.4 


Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools 
and Staffing Survey (Teacher Questionnaire). 


Table C6. Teachers’ race, weighted estimates, 2011/12 

1 Race Percent 

Standard error 

White 89.5 

0.9 

Black 7.8 

0.5 

Other 2.8 

0.8 


Note: Percentages do not sum to 100 because of rounding. 

Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools 
and Staffing Survey (Teacher Questionnaire). 


Teacher professional characteristics 


Table C7. Teachers’ school level taught, weighted estimates, 2011/12 


1 School level 

Percent 

Standard error | 

Elementary 

51.4 

1.3 

Secondary (middle or high school) 

48.6 

1.3 


Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools 
and Staffing Survey (Teacher Questionnaire). 


Table C8. Teachers’ years of teaching experience, weighted estimates, 2011/12 


Teaching experience Mean Standard error 


Years of teaching experience 13.8 0.3 


Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools 
and Staffing Survey (Teacher Questionnaire). 
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Table C9. Teachers’ previous-year evaluation rating, weighted estimates, 2011/12 

1 Evaluation rating 

Percent 

Standard error I 

Excellent, outstanding, or highly effective 

60.3 

1.7 

Satisfactory or effective 

36.8 

1.7 

Unsatisfactory or not that effective 

2.9 

0.7 


Note: Teachers were asked in 2012/13 to report their evaluation ratings from 2011/12. 

Source: Authors' calculations using data from the National Center for Education Statistics 2012/13 Teacher 
Follow-up Survey (Questionnaire for Former Teachers and Questionnaire for Current Teachers). 


School characteristics 


Table CIO. School enrollment characteristics, weighted estimates, 2011/12 


1 Enrollment characteristic 

Mean 

Standard error 1 

Enrollment 

839.5 

25.0 

Percentage of students who are English learner students 

8.4 

0.5 

Percentage of students approved for free or reduced-price lunch 

48.5 

1.0 

Percentage of students with individualized education program 

12.8 

0.6 

Percentage of students who are a racial/ethnic minority 

46.6 

1.2 


Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools 
and Staffing Survey (School Questionnaire). 


Table Cll. Urbanicity of school, weighted estimates, 2011/12 

1 Urbanicity 

Percent 

Standard error 1 

City 

27.7 

1.7 

Suburb 

32.6 

1.7 

Town 

11.7 

1.0 

Rural 

28.1 

1.6 


Note: Percentages do not sum to 100 because of rounding. 

Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools 
and Staffing Survey (Teacher Questionnaire). 
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Logistic regressions 


Table C12. Logistic regression results of school climate and teacher satisfaction with the evaluation 
process, 2011/12 


Coefficient 

Standard error 

Significance 

Odds 

Variable 

a 

6 

(P) 

ratio 

Constant 

-1.00 

0.70 

.16 

0.4 

Independent variables 

Principal leadership 3 

0.91 

0.17 

.00 

2.5*** 

Teacher influence 11 

0.07 

0.20 

.73 

i.i 

Student test scores included in evaluation process 





(compared to not included) 

-1.02 

0.23 

.00 

q 4 *** 

Teacher demographic characteristics 

Female (compared to male) 

0.26 

0.33 

.43 

1.3 

Hispanic (compared to non-Hispanic) 

0.03 

0.57 

.95 

1.0 

Black (compared to White) 

0.29 

0.53 

.58 

1.3 

Other (compared to White) 

0.31 

0.90 

.73 

1.4 

Teacher professional characteristics 

School level taught is elementary (compared to secondary) 

-0.22 

0.31 

.49 

0.8 

Years of teaching experience 

-0.01 

0.01 

.24 

1.0 

Satisfactory or effective evaluation rating in previous year 





(compared to excellent, outstanding, or highly effective rating) 

-0.88 

0.24 

.00 

0.4** 

Unsatisfactory or not that effective evaluation rating in previous 





year (compared to excellent, outstanding, or highly effective rating) 

-5.11 

1.13 

.00 

0 o*** 

Current teacher (compared to former teacher) 

0.18 

0.31 

.57 

1.2 

School characteristics 

Enrollment 



.98 

1.0 

Suburban (compared to city) 

-0.14 

0.36 

.69 

0.9 

Town (compared to city) 

-0.22 

0.38 

.58 

0.8 

Rural (compared to city) 

0.12 

0.36 

.74 

1.1 

Percentage of students who are English learner students 

0.01 

0.01 

.50 

1.0 

Percentage of students approved for free or reduced-price lunch 



.92 

1.0 

Percentage of students with an individualized education program 

0.01 

0.01 

.53 

1.0 

Percentage of students who are a racial/ethnic minority 


0.01 

.61 

1.0 


.. is a negligible value. 

** is significant at p < .01; *** is significant at p < .001. 

a. A composite variable indicating level of agreement (on a scale of 1, strongly disagree, to 4 strongly agree) with six items related to 
principal leadership (see appendix B). 

b. A composite variable indicating teachers’ perception of influence (on a scale of 1, no influence, to 4, a great deal of influence) on 
seven items related to school policy (see appendix B). 

Note: n = 3,080 weighted to represent a population estimate of 2,389,401. Teachers were asked in 2012/13 to report their evaluation 
ratings from 2011/12 and to report their satisfaction with the evaluation process based on their experiences in 2011/12. Model overall 
is significant F(20, 68) = 4.4, Prob > F = 0.0, Design degrees of freedom = 87. Goodness-of-fit test: Hosmer & Lemeshow F(9,79) = 0.5, 
Prob > F = 0.9. 

Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools and Staffing Survey 
(School Questionnaire and Teacher Questionnaire) and 2012/13 Teacher Follow-up Survey (Questionnaire for Former Teachers and 
Questionnaire for Current Teachers). 
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Table C13. Ordinary least squares regression results of school climate and teacher satisfaction with 

the evaluation process with state fixed effects, 2011/12 



Variable 

Coefficient g 

Standard error g 

Significance (p) 

Constant 

0.57 

0.10 

oo*** 

Independent variables 

Principal leadership 3 

0.13 

0.02 

oo*** 

Teacher influence 11 

-0.01 

0.02 

.78 

Student test scores included in evaluation process (compared 
to not included) 

-0.09 

0.04 

.04* 

Teacher demographic characteristics 

Female (compared to male) 

0.20 

0.04 

.59 

Flispanic (compared to non-Hispanic) 

0.02 

0.06 

.77 

Black (compared to White) 

0.03 

0.05 

.57 

Other (compared to White) 


0.10 

.98 

Teacher professional characteristics 

School level taught is elementary (compared to secondary) 

-0.03 

0.03 

.41 

Years of teaching experience 



.20 

Satisfactory or effective evaluation rating in previous year 
(compared to excellent, outstanding, or highly effective rating) 

-0.12 

0.03 

oo*** 

Unsatisfactory or not that effective evaluation rating in previous 
year (compared to excellent, outstanding, or highly effective rating) 

-0.73 

0.07 

oo*** 

Current teacher (compared to former teacher) 

0.03 

0.05 

.48 

School characteristics 

School enrollment 



.86 

Suburban (compared to city) 

-0.02 

0.04 

.65 

Town (compared to city) 

-0.02 

0.05 

.72 

Rural (compared to city) 

0.01 

0.04 

.78 

Percentage of students who are English learner students 



.42 

Percentage of students approved for free or reduced-price lunch 



.66 

Percentage of students with an individualized education program 



.83 

Percentage of students who are a racial/ethnic minority 



.93 


.. is a negligible value. 

* is significant at p < .05; *** is significant at p < .001. 

a. A composite variable indicating level of agreement (on a scale of 1, strongly disagree, to 4 strongly agree) with six items related to 
principal leadership (see appendix B). 

b. A composite variable indicating teachers’ perception of influence (on a scale of 1, no influence, to 4, a great deal of influence) on 
seven items related to school policy (see appendix B). 

Note: n = 3,080 weighted to represent a population estimate of 2,389,401. Teachers were asked in 2012/13 to report their evaluation 
ratings from 2011/12 and to report their satisfaction with the evaluation process based on their experiences in 2011/12. The model 
overall is significant F(70, 12) = 9.3, Prob > F = 0.00, design degrees of freedom = 81. R-squared = 0.29. 

Source: Authors' calculations using data from the National Center for Education Statistics 2011/12 Schools and Staffing Survey 
(School Questionnaire and Teacher Questionnaire) and 2012/13 Teacher Follow-up Survey (Questionnaire for Former Teachers and 
Questionnaire for Current Teachers). 
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Note 


1. According to the 2012 revised statistical standards and guidelines for the National 
Center for Education Statistics, “variance estimates must be calculated by a method 
appropriate to a survey’s sample design (for example, unequal probabilities of selec- 
tion, stratification, clustering, and the effects of nonresponse, post-stratification, and 
raking). These estimates must reflect the design effect resulting from the complex 
design. Approximate variance estimation methods that adjust for most of the impact 
of clustering and stratification include replication methods [bootstrap, jackknife, bab 
anced repeated replication] and Taylor-series linearization. Replication methods can 
also adjust for the impact of nonresponse, post-stratification, and raking” (National 
Center for Education Statistics, 2012, Standard 5). The balanced repeated replication 
method is recommended for use with the Schools and Staffing Survey and Teacher 
Follow-up Survey. 


Notes-1 


References 


Borman, G. D., & Dowling, N. M. (2008). Teacher attrition and retention: A meta-analytic 
and narrative review of the research. Review of Educational Research, 78(3), 376-409. 
http://eric.ed.gov/lkDEJ896364 

Boyd, D., Grossman, P., Ing, M., Lankford, H., Loeb, S., & Wyckoff, J. (2011). The influ- 
ence of school administrators on teacher retention decisions. American Educational 
Research Journal, 48(2), 303-333. http://eric.ed.gov/lkDEJ921700 

Coggshall, J. G., Ott A., & Lasagna, M. (2010). Convergence and contradictions in teachers’ 
perceptions of policy reform ideas. (Retaining Teacher Talent, Report No. 3). Naper- 
ville, IL: Learning Point Associates, and New York: Public Agenda, http://eric.ed.gov/ 
?id=ED508143 

Cohen, J., McCabe, L., Michelli, N. M., & Pickeral, T. (2009). School climate: Research, 
policy, practice, and teacher education. Teachers College Record, 111(1), 180-213. http:// 
eric.ed.gov/?id=EJ826002 

Daley, G., & Kim, L. (2010). A teacher evaluation system that works: A working paper. 
Santa Monica, CA: National Institute for Excellence in Teaching, http://eric.ed.gov/ 
?id=ED533380 

Darling-Hammond, L. (2012). Creating a comprehensive system for evaluating and supporting 
effective teaching. Stanford, CA: Stanford Center for Opportunity Policy in Education. 
http://eric.ed.gov/lkDED532978 

Darling'Hammond, L. (2015). Can value added add value to teacher evaluation? Educa - 
donal Researcher, 44(2), 132-137. http://eric.ed.gov/lkDEJ1054680 

Datnow, A., & Castellano, M. (2000). Teachers’ responses to Success for All: How beliefs, 
experiences, and adaptations shape implementation. American Educational Research 
Journal, 37(3), 775-799. http://eric.ed.gov/lkDEJ627384 

DragO'Severson, E. (2012). New opportunities for principal leadership: Shaping school cli- 
mates for enhanced teacher development. Teachers College Record, 114(3), 1-44. http:// 
eric.ed.gov/?kDEJ1000010 

Goldhaber, D., & Walch, J. (2011). Strategic pay reform: A student outcomes-based evaluation 
of Denver’s ProComp Teacher Pay Initiative (CEDR Working Paper No. 2011-3). Seattle, 
WA: University of Washington. http://eric.ed.gov/lkDED537147 

Goldring, R., Taie, S., & Riddles, M. (forthcoming). User’s manual for the 2012/13 Teacher 
Follow-up Survey current teacher and former teacher data files. (NCES No. 2014-421). U.S. 
Department of Education. Washington, DC: National Center for Education Statistics. 

Goldring, R., Taie, S., Rizzo, L., Colby, D., & Fraser, A. (2013). User’s manual for the 
2011-12 Schools and Staffing Survey, Volume 1: Overview (NCES No. 2013-330). U.S. 
Department of Education. Washington, DC: National Center for Education Statistics. 


Ref-1 


Gordon, R., Kane, T. J., & Staiger, D. O. (2006). Identifying effective teachers using pen 
formance on the job. Washington, DC: The Brookings Institution, http://eric.ed.gov/ 
?id=ED495040 

Grissom, J. A. (2011). Can good principals keep teachers in disadvantaged schools? Linking 
principal effectiveness to teacher satisfaction and turnover in hard-to-staff environ- 
ments. Teachers College Record, 113(11), 2552-2585. http://eric.ed.gov/?id=EJ951114 

Hamilton, L. C. (1990). Modern data analysis: A first course in applied statistics. Belmont, 
CA: Brooks/Cole Publishing Company. 

Heneman, H. G. Ill, Milanowski, A., Kimball, S. M., & Odden, A. (2006). Standards-based 
teacher evaluation as a foundation for knowledge - and skill-based pay (Policy Brief 
No. RB-45). Philadelphia, PA: Consortiumfor Policy Research in Education, http://eric.ed. 
gov/?id=ED493116 

Jiang, J. Y., Sporte, S. E., & Luppescu, S. (2015). Teacher perspectives on evaluation 
reform: Chicago’s REACH students. Educational Researcher, 44(2), 105-116. http://eric. 
ed.gov/?id=EJ1054677 

Johnson, S. M. (2006). The workplace matters: Teacher quality, retention, and effectiveness. 
Washington, DC: National Education Association. http://eric.ed.gov/lkHED495822 

Johnson, S. M., Kraft, M. A., & Papay, J. P. (2012). How context matters in high-need 
schools: The effects of teachers’ working conditions on their professional satisfaction 
and their students’ achievement. Teachers College Record, 114(10), 1-39. http://eric. 
ed.gov/?id=EJ1001989 

Johnson, S. M., Papay, J. P, Fiarman, S. E., Munger, M. S., & Qazilbash, E. K. (2010). 
Teacher to teacher: Realizing the potential of peer assistance and review. Washington, 
DC: Center for American Progress. Retrieved April 22, 2015, from https://www. 
americanprogress.org/wp -content/uploads/issues/2010/05/pdf/par.pdf. 

Kraft, M. A., Marinell, W. H., & Yee, D. (2015). School organizational contexts, teacher turn- 
over, and student achievement: Evidence from panel data. Unpublished working paper. 

Kraft, M. A., & Papay, J. (2014). Can professional environments in schools promote teacher 
development? Explaining heterogeneity in returns to teaching experience. Educational 
Evaluation and Policy Analysis, 36(4), 476-500. http://eric.ed.gov/lkUEJ1046255 

Ladd, H. (2011). Teachers’ perceptions of their working conditions: How predictive of 
planned and actual teacher movement? Educational Evaluation and Policy Analysis, 
33(2), 235-261. http://eric.ed.gov/lkUEJ927621 

McLaughlin, M. W. (1987). Learning from experience: Lessons from policy implementa- 
tion. Educational Evaluation and Policy Analysis, 9(2), 171-178. 


Ref-2 


Measures of Effective Teaching Project. (2010). Learning about teaching: Initial findings from 
the Measures of Effective Teaching project. Seattle, WA: Bill & Melinda Gates Founda- 
tion. http://eric.ed.gov/lkNED528388 

Measures of Effective Teaching Project. (2012). Gathering feedback for teaching: Combining 
high-quality observations with student surveys and achievement gains. Seattle, WA: Bill & 
Melinda Gates Foundation. http://eric.ed.gov/lkNED540960 

National Center for Education Statistics. (2012). 2012 Revisions of NCES Statistical Stan- 
dards: Final. Washington, DC: Institute of Education Sciences. Retrieved September 
22, 2014, from http://nces.ed.gov/statprog/2012/. 

National Council on Teacher Quality. (2012). State of the states 2012: Teacher effectiveness 
policies. Washington, DC: Author. http://eric.ed.gov/lkNED536371 

Riordan, J., LacirenoTaquet, N., Shakman, K., Bocala, C., & Chang, Q. (2015). Re-designing 
teacher evaluation: Lessons learned from a pilot implementation (REL 2015-030). Wash- 
ington, DC: U.S. Department of Education, Institute of Education Sciences, National 
Center for Education Evaluation and Regional Assistance, Regional Educational Lab- 
oratory Northeast & Islands. http://eric.ed.gov/lkNED552484 

Sartain, L., Stoelinga, S. R., & Brown, E. (2011). Rethinking teacher evaluation in Chicago: 
Lessons learned from classroom observations, principal-teacher conferences, and district 
implementation. Chicago, IL: Chicago Consortium for School Research, http://eric.ed. 
gov/?id=ED527619 

Toch, T., & Rothman, R. (2008). Rush to judgment: Teacher evaluation in public education. 
Washington, DC: Education Sector. http://eric.ed.gov/lkNED502120 

U.S. Department of Education. (2011). Guidance on fiscal year 2010 school improvement 
grants. Washington, DC: Office of Elementary and Secondary Education. Retrieved 
June 25, 2012, from http://www2.ed.gov/programs/sif/sigguidance02232011.pdf. 

U.S. Department of Education. (2013). ESEA flexibility. Washington, DC: Author. 
Retrieved July 3, 2013, from http://www2.ed.gov/policy/elsec/guid/esea'flexibility/index. 
html. 

Walsh, E., & Lipscomb, S. (2013). Classroom observations from phase 2 of the Pennsylvania 
Teacher Evaluation Pilot: Assessing internal consistency, score variation, and relation- 
ships with value added: Final report. Cambridge, MA: Mathematica Policy Research. 
Retrieved April 24, 2015, from http://www.mathematica'mpr.com/~/media/publica' 
tions/pdfs/education/classroom_observations.pdf. 

Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: Our national 
failure to acknowledge and act on differences in teacher effectiveness. Brooklyn, NY: The 
New Teachers Project. http://eric.ed.gov/lkNED515656 


Ref-3 


The Regional Educational Laboratory Program produces 7 types of reports 



Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 


Stated Briefly 

Summaries of research findings for specific audiences 


Applied Research Methods 

Research methods for educational settings 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 




